Applying Judgment Analysis Theory and Methods to Obtain an Insight Into Clinical Judgments: Implementation and Findings With a Simulated Neonatal Intensive Care Unit Setup.

INTRODUCTION: Clinicians who provide acute care are required to quickly identify and judge the illness severity of patients who experience deterioration in their clinical state. Accuracy of judgments can only be tested with respect to a valid reference, but in most health care areas, there is no such score. Judgment analysis theory and methods are presented and proposed as a framework to obtain insight into clinical judgments. A study in a simulated neonatal intensive care unit setup is described to demonstrate the applicability of the proposed methodology. METHODS: Sixteen participants from a neonatal intensive care unit department reviewed 31 clips of simulated cases. The participants were directed to use a 5-point scale to rate their personal interpretation regarding the illness severity of the simulated patient. Judgment analysis techniques were used to identify the judgment capabilities of the participants and to determine factors that influence those capabilities. RESULTS: Most participants interpreted the clinical signs information consistently, but interpretation varied remarkably between clinicians, providing possible explanation to the differences between the clinicians' judgments. Significant correlations were found between the doctors' years of medical experience and attributes of their judgments. CONCLUSIONS: Judgment analysis can be used to obtain insight into clinical judgments and to identify and quantify factors that affect clinicians' judgments. Judgment analysis can promote health care by enhancing clinical assessment teaching, by providing objective and personalized feedback to team members about their judgment performance, and by introducing a unified and objective method to study elements that affect clinical judgments.

Time without ventilation during intubation in neonates as a patient-centred measure of performance.

AIMS: Time without ventilation is often much longer than an intubation attempt, yet patient stability relies on effective gas exchange. We argue that in addition to existing performance criteria, intubation performance measures should include interruption to effective ventilation. METHODS: We reviewed video recorded resuscitations of 31 term and preterm newborns that included at least one intubation attempt. Time stamps were recorded at the end of mask ventilation, laryngoscope insertion and removal (laryngoscope duration), and re-commencement of ventilation via mask or endotracheal tube (ETT). Intubation attempts were defined as Successful (subsequent ventilation via ETT), or Failed (ETT incorrectly placed) or Withdrawn (laryngoscope removed before ETT insertion attempt). RESULTS: During intubation, total time without ventilation varied from 31 to 273s, compared to laryngoscope duration of 12-149s. Time without ventilation as Median [min-max] was greater for failed attempts 64 [48-273]s, yet laryngoscope duration was shortest for failed attempts 33 [21-46]s. Time between ceasing ventilation and commencing intubation was 5 [1-46]s suggesting room for improvement during transitions within the procedure. CONCLUSIONS: Time without ventilation is a more physiologically important measure of a resuscitation team's intubation expertise than laryngoscope duration. Since successful attempts took longer than failed attempts, emphasising haste during vocal cord visualisation and tube insertion may reduce success rates. Reducing the time without ventilation at either end of the procedure may be achievable with better team coordination and could be just as important to patient wellbeing as technical precision.

The accuracy of clinical assessments as a measure for teamwork effectiveness.

INTRODUCTION: Team training in healthcare is usually evaluated by observers who either score trainees' behaviors, social skills, and cognitive skills during simulation or measure changes in the clinical state of a mannequin. Both methods have shortcomings that limit their usefulness. We propose Brunswik's probabilistic functionalism and the Accuracy Score (AS), a measure emerging from judgment analysis, as elements of a complementary approach that could increase the objectivity of team training evaluation. We report an initial investigation. METHOD: Three groups of neonatal clinicians participated in a resuscitation experiment involving three different training interventions. During the experiment, at various phases, the participants were required to assign an Apgar score to a mannequin. ANALYSIS: The AS was used to test how accurately the clinicians assigned Apgar scores to the mannequin across different levels of task demand, training content, and training delivery method. RESULTS: The AS was lower when task demand increased (P < 0.01). The AS was higher after teamwork training than after clinical training (P < 0.05) and better after hands-on teamwork training than after lecture-based teamwork training (P < 0.05). CONCLUSIONS: Because it is simple and objective, the AS may complement existing measures for team training evaluation. Future studies are required in which the AS is tested with a larger number of trainees, in longitudinal experiments, across different training areas, and is compared with previously validated team performance measures.

Presenting video recordings of newborn resuscitations in debriefings for teamwork training.

BACKGROUND: The Neonatal Resuscitation Program (NRP) and similar courses have been used to train clinicians. However, formal teamwork training was not included in these courses, and their effectiveness has been questioned. In adult resuscitation, debriefings using video recordings have improved outcomes, but recordings of neonatal resuscitation have been used primarily for research. AIMS: To test if debriefings that include video recordings of neonatal resuscitations can improve teamwork and clinical practice. METHODS: Over 9 months, clinicians voluntarily attended weekly debriefings in which recently performed resuscitations were presented, evaluated and then discussed. Discussions were focused on teamwork and were facilitated by an education nurse and a human factors expert with a training background. Subsequently, three experts, a neonatologist, a neonatal nurse educator and a midwifery educator from different organisations viewed and scored 19 recordings from the first 4.5 months and 19 recordings from the second 4.5 months. Experts were blinded to the recording period. The two sets of recordings were balanced for babies' gestation and birth weight. Scoring included 'teamwork' items, 'following guidelines' items and 'temporal control of the resuscitation procedure' items. RESULTS: All 'teamwork' items improved between the two periods and one item improved significantly (p<0.05). Scores for 'Following guidelines' showed little change except 'Intubation' which deteriorated (p<0.05). There was no significant interaction between periods and raters. Cronbach's alpha indicated inter-expert rating consistency ranging from 0.54 to 0.86. CONCLUSIONS: Voluntary debriefings had some positive effect on neonatal resuscitation teamwork. Future studies are warranted to determine the best methods for debriefing combined with other training methods.

Clinicians can accurately assign Apgar scores to video recordings of simulated neonatal resuscitations.

INTRODUCTION: The Apgar score is used to describe the clinical condition of newborns. However, clinicians show low reliability when assigning Apgar scores to video recordings of actual neonatal resuscitations. Simulators provide a controlled environment for recreating and recording resuscitations. Clinicians assigned Apgar scores to such recordings to test the representativeness of simulator and recordings. Study design was guided by Brunswik's probabilistic functionalism. METHOD: Judgment analysis methods were used to design 51 recordings of neonatal resuscitation scenarios, simulated with SimNewB (Laerdal, Stavanger, Norway). A step-by-step explanation of the design, preparation, and testing of the recordings is provided. ANALYSIS: Recorded Apgar scores, calculated from the presentation of clinical signs, were compared against the designed scores. Working independently and without feedback, three experts assigned Apgar scores to confirm that the recordings could be interpreted as intended. Seventeen neonatal resuscitation clinicians scored the recordings in a separate experiment. RESULTS: Correlations between Apgar scores assigned by the 20 viewers (experts plus clinicians) and recorded Apgar scores were high (0.78-0.91) and significant (P < 0.01). Fourteen of the 20 viewers scored the recordings without significant bias. Correlations between viewers' scores and scores of individualized linear models calculated for each viewer were high (0.79-0.97) and significant (P < 0.01), indicating systematic judgments. CONCLUSIONS: SimNewB provided a realistic presentation of clinical conditions that was preserved in the recordings. Clinicians could interpret clinical conditions systematically and accurately without feedback or detailed instructions. These methods are applicable to future research about accuracy of clinical assessments in actual and simulated environments.